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DETAILED ACTION 

Response to Amendment 

1 . Applicant's amendments filed on October 7, 2009 have been entered. Claims 1- 
3, 14, and 16 have been amended. Claims 4, 11-13, 15, and 17 have been canceled. 
No claims have been added. Claims 1-3, 5-10, 14, and 16 are still pending in this 
application, with claims 1 and 14 being independent. 

Claim Objections 

2. Claims 1 and 14 are objected to because of a minor typographical error: at the 
end of the calculation limitations, the claims recite "included the virtual object". 
Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 

forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-3, 7, 8, 14, and 16 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kato et al. (U.S. Patent No. 5,999,185), referred herein as Kato, in 
view of Arbter et al. (U.S. Patent No. 6,545,663), referred herein as Arbter. 
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Regarding claim 1, Kato teaches an image processing method of generating a 
display image of a virtual space including a virtual object consisting of at least one part 
(abstract; column 6, lines 50-52), comprising: a first acquisition step of acquiring a 
position and orientation of a viewpoint of an observer (column 2, lines 38-50; column 17, 
lines 1-2); a second acquisition step of acquiring a position and orientation of a pointing 
device (figure 1, pointing device 3A; column 10, lines 53-55; column 17, lines 1-7); a 
calculation step of calculating a position of an object in the virtual space based on the 
positions of the viewpoint and the pointing device, wherein the calculated position of the 
object is near the position of the pointing device and closer to the position of the 
viewpoint than that of the pointing device (column 2, lines 38-50; column 17, lines 1-2, 
11-13, and 26-28; column 28, lines 59-64; column 29, lines 3-7), and displaying a list 
image in the virtual space that represents a list of pieces of information about one or 
more parts, included in the virtual object, near the position of the pointing device; a 
layout step of laying out the list image in the virtual space (figure 18; figure 24; figures 
25A, B, and C; column 6, lines 50-52; column 20, lines 27-44; column 21, lines 45-47 
and 53-56; column 22, lines 62-66; column 23, lines 2-4, 18-23, 31-34, and 47-50); a 
virtual space image generation step of generating the display image of the virtual space, 
in which the laid out list image and the virtual object are included, on the basis of the 
acquired position and orientation of the viewpoint (Kato, column 2, lines 38-50; column 
17, lines 1-2; column 20, lines 36-40; column 28, lines 59-64); and a composition step of 
composing the generated display image of the virtual space and an image of a physical 
space seen in accordance with the position and orientation of the viewpoint to display 
the composed image at a head mounted display mounted on the observer's head 
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(column 15, lines 58-62; column 17, lines 11-13 and 26-28; column 20, lines 36-40; 
column 25, lines 48-51; column 28, lines 59-64; column 29, lines 22-26). Kato does not 
teach the method, wherein the list image is positioned at the calculated position in the 
virtual space. Arbter teaches a method for processing the position of an object to be 
displayed in a virtual space, comprising a calculation step that calculates a position of 
an image of an object in the virtual space based on the positions of a viewpoint and a 
pointing device, wherein the calculated position in near the position of the pointing 
device (abstract, lines 1-6 and 15-19; column 1, lines 6-9 and 12-16; column 3, lines 15- 
20; column 4, lines 58-62; column 6, lines 16-25). As taught by Arbter, utilizing this 
position calculation enables a user to present visual information pertaining to the virtual 
objects in an easy and intuitive manner, and provides for efficient interaction with 
objects in the virtual space. Therefore, it would have been obvious to a person having 
ordinary skill in the art at the time the invention was made to combine the position 
calculation disclosed by Arbter with the invention disclosed by Kato. 

Regarding claim 2, Kato in view of Arbter teaches the method according to claim 
1 , and further teaches the method, wherein in the layout step, the list image is laid out at 
the orientation of the viewpoint (Kato, figure 18; figures 24A, B, and C; column 20, lines 

27- 44; column 21, lines 45-47 and 53-56; column 22, lines 62-66; column 23, lines 2-4, 
18-23, 31-34, and 47-50; Arbter, column 4, lines 58-62; column 6, lines 16-25). 

Regarding claim 3, Kato in view of Arbter teaches the method according to claim 
1 , and further teaches the method, wherein in the calculation step, the position of the list 
image is calculated to be a position that internally divides a line segment connecting 
between the positions of the viewpoint and the pointing device (Kato, column 20, lines 

28- 32 and 47-51; column 36, lines 35-49). 
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Regarding claim 7, Kato in view of Arbter teaclies tine method according to claim 
1 , and further teaches the method, characterized by further comprising a distance 
calculation step of calculating a distance between the position of the pointing device and 
the position of the virtual object (Kato, figures 35 and 36; column 29, lines 3-17), and a 
list image generation step of generating the list image to display the list of pieces of 
information about the parts up to a layer level corresponding to the distance calculated 
in the distance calculation step in a hierarchical structure of the parts included in the 
virtual object (Kato, figure 6; figures 48A, B, and C; column 1 1 , lines 58-67; column 12, 
lines 1-7; column 35, lines 23-46). 

Regarding claim 8, Kato in view of Arbter teaches the method according to claim 
1 , and further teaches the method, characterized in that the list image is an image to 
display a list of pieces of information about, of the parts included in the virtual object, a 
part at a position closest to the position of the pointing device acquired in the second 
acquisition step (Kato, column 17, lines 1-7 and 11-13; column 20, lines 27-44; column 
21, lines 45-47 and 53-56; column 22, lines 62-66; column 23, lines 2-4, 18-23, 31-34, 
and 47-50). 

Regarding claim 14, the limitations of this claim correspond to the limitations of 
claim 1 ; thus, they are rejected on the same grounds as the limitations of claims 1 . 

Regarding claim 16, Kato in view of Arbter teaches a computer-readable storage 
medium having a program stored therein, said program being characterized by causing 
a computer to execute an image processing method of claim 1 (Kato, figure 2, storage 
medium 7; figure 3, main program 18; column 10, line 67; column 11, lines 1-2; column 
16, lines 50-51). 
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5. Claims 5 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kato in view of Arbter, and further in view of Buxton et al. (U.S. Patent No. 6,1 18,427), 
referred herein as Buxton. 

Regarding claim 5, Kato in view of Arbter teaches the method according to claim 
1 , but does not teach the method, characterized by further comprising a conversion step 
of converting the list image into a semitransparent image in accordance with an 
instruction to change the list image to the semitransparent image. Buxton teaches a 
method for optimal transparency processing in a graphical user interface to merge 
graphical images of objects, comprising conversion of the images into semitransparent 
images in accordance with an instruction to change the images to the semitransparent 
image (column 3, lines 40-46). As was very well known to those of ordinary skill in the 
art, and illustrated by Buxton, image transparency enables a user to view multiple layers 
of information without losing information that would otherwise be occluded by 
overlapping images, thus increasing the efficiency and simplicity of the user interaction, 
as well as improving the quality of the imaging system as a whole. Therefore, it would 
have been obvious to a person having ordinary skill in the art at the time the invention 
was made to combine the semitransparent images of Buxton with the invention 
disclosed by Kato in view of Arbter. 

Regarding claim 6, Kato in view of Arbter teaches the method according to claim 
1 , and further teaches the method, characterized by further comprising a determination 
step of determining on the basis of the position and orientation of the viewpoint and a 



Application/Control Number: 1 0/581 ,51 4 Page 7 

Art Unit: 2628 

position of tine virtual object wlietlier tine virtual object is present in a direction of line of 
sight of the viewpoint (Kato, column 17, lines 1-7; column 40, lines 55-63). Kato in view 
of Arbter does not teach the method, further comprising a transparency control step of, 
when it is determined in the determination step that the virtual object is present, making 
a transparency of the list image higher than that when it is determined in the 
determination step that the virtual object is not present. Buxton teaches a method for 
optimal transparency processing in a graphical user interface to merge graphical 
images of objects, comprising a transparency control step of, when it is determined that 
an object is present, making a transparency of an image higher than that when it is 
determined that an object is not present (column 3, lines 40-46; column 6, lines 58-65). 
As was very well known to those of ordinary skill in the art, and illustrated by Buxton, 
image transparency enables a user to view multiple layers of information without losing 
information that would otherwise be occluded by overlapping images, thus increasing 
the efficiency and simplicity of the user interaction, as well as improving the quality of 
the imaging system as a whole. Therefore, it would have been obvious to a person 
having ordinary skill in the art at the time the invention was made to combine the 
transparent images of Buxton with the invention disclosed by Kato in view of Arbter. 

6. Claims 9 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kato in view of Arbter, and further in view of Stroyan (U.S. Patent No. 5,923,333), 
referred herein as Stroyan. 



Application/Control Number: 1 0/581 ,51 4 Page 8 

Art Unit: 2628 

Regarding claim 9, Kato in view of Arbter teaclies tine method according to claim 
1 , and further teaches the method, characterized in that in the virtual space image 
generation step, when the list image overlaps a hand region in the physical space 
image acquired in the physical space image acquisition step (Kato, column 12, lines 18- 
26). Kato in view of Arbter does not explicitly teach the method, wherein the image of 
the virtual space Is generated on the basis of priority data to designate which of the 
hand region and the list image should be rendered in front. Stroyan teaches a method 
for fast alpha transparency rendering, wherein an image of a virtual scene Is generated 
on the basis of priority data to designate which of a plurality of virtual objects should be 
rendered In front (column 1, lines 57-67; column 2, lines 1-5). As was well known at the 
time of the invention, and illustrated by Stroyan, utilizing priority data to render objects in 
a virtual scene results in an accurate representation of the virtual scene, and 
accomplishes this in a fast and efficient manner. Therefore, it would have been obvious 
to a person having ordinary skill in the art at the time the invention was made to 
combine the priority rendering method of Stroyan with the invention disclosed by Kato in 
view of Arbter. 

Regarding claim 10, Kato in view of Arbter, further in view of Stroyan teaches the 
method according to claim 9, and further teaches the method, characterized by further 
comprising a designation step of designating which of the hand region and the list 
image should be rendered in front, wherein in the designation step, designated contents 
are set to the priority data (Stroyan, column 1, lines 57-67; column 2, lines 1-5). 
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Response to Arguments 

7. Applicant's arguments, see page 8, filed October 7, 2009, with respect to the 
claim objections have been fully considered and are persuasive. The amendments to 
the claims are sufficient to overcome the informalities of the original claims; thus the 
objections to these claims have been withdrawn. 

8. Applicant's arguments, see page 8, filed October 7, 2009, with respect to the 1 1 2 
rejections have been fully considered and are persuasive. The amendments to the 
claims are sufficient to overcome the 1 12 rejections; thus the 1 12 rejections of these 
claims have been withdrawn. 

9. Applicant's arguments with respect to claim 1 have been fully considered, but 
they are not persuasive. On page 9 of the Applicant's Remarks, the Applicant argues 
that Kato does not teach the claimed subject matter because 1 ) the position of menu 37 
of Kato is not in the virtual space, and 2) menu 37 is at a fixed position, and is not 
calculated according to a position of a viewpoint of the user. Regarding the first 
argument, the Examiner respectfully submits that the Office Action relies on the list 
images of figures 25A-C, which are disclosed by Kato as being displayed in the virtual 
space. However, and in regard to the second argument, as discussed in the above 
Office Action, Kato does not disclose the amended limitation that the list image is 
positioned based on the calculated position. This, however, is moot in view of the new 
ground(s) of rejection. 
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Conclusion 

10. The following prior art made of record and not relied upon is considered pertinent 
to applicant's disclosure: 

Neumann et al. (U.S. Patent Application Publication No. 2002/0191862); 

Augmented-reality tool employing scene-feature auto-calibration during 
camera motion. 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DAVID T. WELCH whose telephone number is 
(571)270-5364. The examiner can normally be reached on Monday-Thursday, 8:00- 
5:30 EST, and alternate Fridays, 8:00-4:30 EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Xiao Wu can be reached on (571 )272-7761 . The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/dtw/ 

/XIAO M. WU/ 

Supervisory Patent Examiner, Art Unit 2628 



